debugHelper--trace prog's OpenGL calls by overriding DSO fxns; DT: src/exampleCode/opengl/debugHelper

debugHelper OpenGL "debugger" utility

HUB | Up | Pheedbak | Download | Tree | Topic | A-Z | Search | Hot | New

debugHelper overrides DSO functions to trace a program's OpenGL calls
without recompiling

NOTE:  this has nothing whatsoever to do with the actual OpenGL
       Debug application implemented by the OpenGL group and
       released with IRIX 5.3.  we continue to include it here
       in the d.t. as a useful dso-opengl debugging device.

This is an "OpenGLDebug" used by the Inventor group when developing Open Inventor. It helped find lots of bugs in both Open Inventor and OpenGL. It prints out a trace of the OpenGL calls made by the program as it runs.

It does not intercept all OpenGL calls, only the subset of OpenGL (and GLU and GLX) calls made by Open Inventor. However, it is easy to add routines that are missing.

The technique used to intercept the OpenGL calls made by any application, call a different routine, and then have that routine call the real OpenGL routine, is very powerful. Using this technique, you can override any public routine in any DSO, for any non-setuid program. The possibilities for Deep Hacks are endless.

Using This as an stand-in OpenGLDebug pretender:
--------------------------------------
First, you must compile OpenGL.c into a DSO:

   
cc -O -c OpenGL.c
ld -shared OpenGL.o -o libGL.so

Next, you must tell the run-time linker to look for DSO's in the current directory before /usr/lib:

LD_LIBRARY_PATH=. ; export LD_LIBRARY_PATH  # If using sh/ksh:

setenv LD_LIBRARY_PATH .                    # If using csh/tcsh:

Then, run any OpenGL program. As an example:

ivview -pq ../../inventor/inventorTemplates/models/queen.iv

(NOTE: ivview hails from the "inventor_eoe.sw.inventor" subsystem)

By default, OpenGL.c produces lines that look like:

OPENGL: glEnable(GL_LIGHT0); If the GLCODE environment variable is set to anything, then the OPENGL: prefix is removed, producing compilable code (for most calls; some calls don't produce compilable code). OpenGL.c will do a reasonable job of producing compilable code if you set the GLCODE environment variable.

Also note that some OpenGL routines call other OpenGL routines to do their work (e.g. glXChooseVisual calls glXGetConfig); these are tagged by their recursion level instead of 'OPENGL' when printed.

How It Works
-----------
First, we write a routine with the same arguments and result as the real GL routine:

   
    GLuint
    glGenLists(GLsizei range)
    {
    	... print out debug info ...
    
    	return /* ??? return something ??? */;
    }

In OpenGL.c, the print stuff gets pretty fancy--gl_header() keeps track of indentation, gl_lookup_enum() translates from enumerated constants to a human-readable string, etc. But that is all pretty straightforward.

The tricky part is actually arranging to call the real glGenLists() routine. We obviously can't just call it directly, because the compiler would interpret that as a recursive call to our replacement glGenLists() routine.

Instead, we use the dlopen() and dlsym() routines. dlopen() allows us to explicitly open up /usr/lib/libGL.so, the real OpenGL DSO. Then dlsym() lets us find the real routines in the real DSO, and returns a pointer to the real routine. We can then call the real routine through that pointer, passing in any arguments and noting any result. Written out for glGenLists(), it would look something like:

   
    void
    glGenLists(GLenum mode)
    {
    	int result;
    
    	printf("glGenLists(%d)\n", mode);
    
    	void *glDSO = dlopen("/usr/lib/libGL.so", RTLD_LAZY);
    	/* check for error... */
    	void (*real_glGenLists)(GLenum) = dlsym(glDSO, "glGenLists");	
    	/* check for error... */
    
    	result = (*real_glGenLists)(mode);
    
    	dlclose(glDSO);
    
    	return result;
    }

Because the code for all the GL routines would look very, very similar, and because it would be really slow to call dlopen()/dlsym()/dlclose() for every GL routine, this DSO magic is packaged up into some handy macros that all the routines call. The macros arrange to:

only dlopen() libGL.so once
only lookup a routine using dlsym() once

Most of the code at the top of OpenGL.c could be adapted for any other DSO in the system; just write appropriate versions of GL_GETSYM, GL_CALL and GL_RCALL that lookup symbols in another DSO.

Writing a preprocessor that reads a .h file and automatically generates 'interception' code is left as an exercise for the reader, as is writing code that actively trys to find OpenGL errors (e.g. passing NaN or Infinity as arguments, calling glPopMatrix too many times, etc).

OR, ...